Figure 2: Anexampleofadynamic (left) andstatic (right) neuralgraph. Claim: Assume L is Lipschitzcontinuous.

The success of neural networks has driven a shift in focus from feature engineering to architecture engineering. However, successful networks today are constructed using a small and manually defined set of building blocks. Even in methods of neural architecture search (NAS) the network connectivity patterns are largely constrained. In this work we propose a method for discovering neural wirings. We relax the typical notion of layers and instead enable channels to form connections independent of each other.

Neural Information Processing Systems http://nips.cc/

The core value of the submission is in the idea and formulation. This is perhaps somewhat subjective, and so hard it is hard to describe the supporting evidence, but it is an especially elegant description of the network design problem. The unification of core parts of the sparse neural network literature combined with the neural architecture search problem is very nice, and will mean that the paper of interest to a very large cross-section of the NeurIPS community. The setup of the comparisons is limited in some ways, but smaller choices were clearly made well. For instance, in the negative, only a few sample networks are presented.

The paper presents an interesting method to efficiently learn channel-wise wirings in deep neural networks, with compelling results in the low FLOPS regime. It has practical value and should be of wide interest to the NeurIPS community. All reviewers recommend acceptance, and the AC agrees with this decision.

The success of neural networks has driven a shift in focus from feature engineering to architecture engineering. However, successful networks today are constructed using a small and manually defined set of building blocks. Even in methods of neural architecture search (NAS) the network connectivity patterns are largely constrained. In this work we propose a method for discovering neural wirings. We relax the typical notion of layers and instead enable channels to form connections independent of each other.

The success of neural networks has driven a shift in focus from feature engineering to architecture engineering. However, successful networks today are constructed using a small and manually defined set of building blocks. Even in methods of neural architecture search (NAS) the network connectivity patterns are largely constrained. In this work we propose a method for discovering neural wirings. We relax the typical notion of layers and instead enable channels to form connections independent of each other.